Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures. RNNs, widely used to process time series and sequence information, could greatly benefit from complex representations. We present a novel complex gated recurrent cell, which is a hybrid cell combining complex-valued and norm-preserving state transitions with a gating mechanism. The resulting RNN exhibits excellent stability and convergence properties and performs competitively on the synthetic memory and adding task, as well as on the real-world tasks of human motion prediction.

Summary of approach and contributions: The authors resurrect the pioneering work of Hirose on complex valued neural networks in order to provide a new RNN based on a complex valued activation/transition function and a complex argument gating mechanism. In order to obtain a differentiable function that is not constant and yet bounded, the authors step away from holomorphic functions and employ CR calculus. The authors show experimental improvements on two synthetic tasks and one actual data set. Strengths of the paper: o) Moving away from strict holomorphy and using CR calculus to apply complex valued networks to RNNs is interesting as a novel technique. I think that the authors should spend more time explaining how phases can be easily encoded in the complex domain and therefore why such complex representations can be advantageous for sequential learning.

Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures. RNNs, widely used to process time series and sequence information, could greatly benefit from complex representations. We present a novel complex gated recurrent cell, which is a hybrid cell combining complex-valued and norm-preserving state transitions with a gating mechanism. The resulting RNN exhibits excellent stability and convergence properties and performs competitively on the synthetic memory and adding task, as well as on the real-world tasks of human motion prediction. Papers published at the Neural Information Processing Systems Conference.